Skip to content

CP-53658: Claim memory on a single NUMA node #6369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 3, 2025

Conversation

psafont
Copy link
Member

@psafont psafont commented Mar 18, 2025

This PR allows xenopsd to launch domains that are using memory allocated in a single NUMA node in a reliable way. (when restarting a VM in a host with most of the memory in use)

Since this is only supported by Xen with this patchqueue: https://lists.xen.org/archives/html/xen-devel/2025-03/msg01127.html, and it needs support from xenguest and emu-manager, there are two commits that revert the changes to just claim memory pages without a NUMA memory node. These are easily revertable by patches to enable this functionality in cutting edge builds.

Revertable patches:

  • xenctrl: Don't use numa_node in domain_claim_pages calls
  • xenopsd/xc: Do not try to allocate pages to a particular NUMA node

@psafont psafont force-pushed the private/paus/numa-xs8 branch from e5fef8f to b3cb4dd Compare March 18, 2025 17:06
@psafont psafont force-pushed the private/paus/numa-xs8 branch 8 times, most recently from c5b6e1f to 905f18c Compare March 26, 2025 16:35
@psafont
Copy link
Member Author

psafont commented Mar 27, 2025

The call claim_pages fails when ballooning is involved and most of the memory in the host is being used, it needs a few changes to ignore these errors and plow ahead, just like before.

@psafont psafont force-pushed the private/paus/numa-xs8 branch from 905f18c to 886e0a6 Compare March 27, 2025 14:56
@psafont
Copy link
Member Author

psafont commented Apr 1, 2025

Tested both in XS8 and XS9 with the two reverted patches, and it works:

  • XS8: toolstack validation and smoke test show that the patches fixes best-effort with DMC. Run 215279
  • XS9: toolstack validation tests show that the numa node parameter is passed to xenguest and emu-manager when starting and resuming domains, respectively. Run 215335

I've also manually tested that the VM that restart works.

I've seen that parallel starts are are also fixed

@psafont psafont marked this pull request as ready for review April 1, 2025 16:30
@psafont psafont force-pushed the private/paus/numa-xs8 branch from 886e0a6 to ed010a1 Compare April 2, 2025 12:30
@robhoes
Copy link
Member

robhoes commented Apr 2, 2025

Use Otion.x directly to avoid any misunderstnadings

In spite of the spelling, I understood this sentence.

@psafont psafont force-pushed the private/paus/numa-xs8 branch 2 times, most recently from a380f3a to 221dc53 Compare April 2, 2025 14:31
psafont added 7 commits April 3, 2025 13:44
Many years ago, 'maybe' used to be a synonym for Option.iter, Option.map and
other functions. Use Option.x directly to avoid any misunderstnadings

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
…ifetime

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Now the numa node needs to be passed. A special value of -1n is used to signify
that no node is meant to be used. Since this is arch-dependent, a Nativeint.t
is used to encode the value.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This binding is only available in Xen 4.21 (unreleased)

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Previously unit was returned every single time, but the result of the inner
function will need to be used in the near future.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Xen currently supports to modes to claim memory for a domain: without any node
in particular, or claim memory in a single NUMA node.

When planning a domain, return the nodes that will host the domain, and how
much memory. In the case where the domain fits in a single NUMA node, claim
pages on that node, otherwise fall back to previous behaviour.

The memory claims need to happen while the memory measurements hold valid, that
is while no VMs are started, otherwise ENOMEM might be returned.

Because the current mode is a best-effort, log when the claim does not work.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Neither xenguest nor emu-manager support passing the parameter just yet, so
avoid passing the numa node to create the parameter.

On top of that claiming memory conflicts with DMC, so it's better to keep
previous behaviour of not claiming any pages before allocating.

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
@psafont psafont force-pushed the private/paus/numa-xs8 branch from 221dc53 to 9e6fb15 Compare April 3, 2025 13:36
Copy link
Member

@robhoes robhoes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@psafont psafont added this pull request to the merge queue Apr 3, 2025
Merged via the queue into xapi-project:master with commit e21647c Apr 3, 2025
17 checks passed
@psafont psafont deleted the private/paus/numa-xs8 branch April 3, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants